1. Field of the Invention
This invention relates to computer graphics and, more particularly, to compositing image elements to form a composite image.
2. Description of Related Art
Many applications of computer graphics involve compositing, the process of assembling two or more image elements to form a composite image.
An important application of compositing is to place an image of an actor over an image of a background environment. The result is that the actor, originally filmed in a studio, appears to be in the real-world location or a completely virtual environment.
For the results to appear to be real, the various elements should usually appear to have been photographed at the same time, in the same place, and with the same camera
Achieving realistic composites requires matching many aspects of the foreground and background elements. The elements must be viewed from consistent perspectives, and the boundary between the two elements must match the contour of the actor, blurring appropriately when the actor moves quickly.
The two images also need to exhibit the same imaging system properties: brightness responds curves, color balance, sharpness, lens flare and noise.
Achieving the final level of realism involves matching the illumination between the foreground and the background: the elements should exhibit the same shading, highlights, indirect illumination, and shadows that they would have had if they had been in the background environment.
If an actor is composited into a cathedral, for example, his or her illumination should appear to come from the cathedral's candles, altars, and stained glass windows. If the actor is walking through a science laboratory, she should appear to be lit by fluorescent lamps, blinking readouts, and indirect lights from the walls, ceiling, and floor. In wider shots, the actor must also photometrically affect the scene: properly casting shadows and appearing in reflections.
The art and science of compositing has produced many techniques for matching these elements, but the one that remains the most challenging has been to achieve consistent and realistic illumination between the foreground and background.
The fundamental difficulty is that, when a person is actually in an environment, it is the environment that illuminates him. When a person is filmed in a studio, he is illuminated by something often quite different—typically a set of studio lights and the green or blue backing used to separate him from the background. When the lighting on the actor is different from the lighting that he would have received in a desired environment, the composite can appear as a collage of disparate elements, rather than an integrated scene from a single camera, breaking the sense of realism and the audience's suspension of disbelief.
Experienced practitioners make their best effort to arrange the on-set studio lights in a way that approximates the positions, colors, and intensities of the direct illumination in a desired background environment; however, the process is time-consuming and difficult to perform accurately. As a result, many digital composites require significant image manipulation by a compositing artist to convincingly match the appearance of the actor to his background.
Complicating this process is that, once the foreground element of an actor is shot, the extent to which a compositor can realistically alter the form of its illumination is relatively restricted. As a result of these complications, creating live-action composites is labor intensive. It also sometimes fails to meet the criteria of realism desired by the filmmakers.